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Assessing Treatment Integrity in Behavioral Consultation 

Lee A. Wilkinson 
Abstract 

The trend in school psychology services is a shift from an emphasis on an assessment-based paradigm to 
one of consultation problem-solving and behavioral intervention. A critical component of consultation- 
derived interventions and behavior change is treatment integrity. Treatment integrity (or fidelity) refers to 
the extent to which an intervention is implemented as intended (or planned). Although its importance has 
been acknowledged in the literature, this construct has largely been neglected in consultation research and 
practice. This article describes practical approaches for assessing and monitoring the integrity of 
treatments implemented during the problem-solving process. A treatment-monitoring interview (TMI) is 
proposed as an initial step towards developing a technology to assess and facilitate treatment integrity in 
behavioral consultation practice. Limitations, implications for practice, and future research directions are 
discussed. 
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The role of consultation in school psychology is expected to expand significantly over the next 
decade as the field experiences a shift from an emphasis on assessment-related activities to an ecological 
and problem solving paradigm of practice (Sheridan & Gutkin, 2000). Surveys consistently indicate that 
this indirect model of service delivery is one of the most preferred and valued professional activities of the 
school psychologist (Gilman & Gabriel, 2004; Kratochwill & Stoiber, 2000; Reschly, 2000). Legislative 
initiatives such as the Individuals with Disabilities Education Reform Act (IDEA '04) and No Child Left 
Behind Act (NCLB) are expected to provide further impetus and support for the expansion of problem- 
solving consultation and intervention-oriented services by school psychologists. 

Behavioral Consultation 

Decades of research have documented the effectiveness of consultation as a frame work for 
delivering services to children with a wide variety of learning and behavioral problems in general 
education settings (Gutkin, 1996; Kratochwill & Stoiber, 2000). Of the several theoretical models 
available to practitioners (e.g., mental health, organizational, and behavioral), behavioral consultation 
(Kratochwill & Bergan, 1990) is the most popular and frequently practiced type of school-based 
consultation. (Erchul & Martens, 2002). The behavioral consultation model has received substantial 
empirical support in reviews, meta-analyses, and case studies (Bramlett & Murphy, 1998; Kratochwill, 
Elliott, & Busse, 1995; MacLeod, Jones, Somers, & Havey, 2001; Medway & Updyke, 1985; Sheridan, 
Welch, & Orme, 1996; Wilkinson, 1997, 2003). 

Treatment Integrity 

Identifying an effective treatment is a necessary but inadequate condition for producing behavior 
change (Wickstrom, Jones, LaFleur, & Witt, 1998). The success of intervention plans developed and 
implemented during behavioral consultation is largely dependent on the extent to which they are 
implemented as intended or planned by consultees (e.g., teachers, parents), or what has been termed 
treatment integrity (Gresham, 1989; Truscott, Richardson, Cohen, Frank, & Palmeri, 2003). Treatment 
integrity (or fidelity) reflects the accuracy and consistency with which each component of the treatment or 
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intervention plan is implemented. It is an important link between the use and effectiveness of 
interventions in school settings and a key aspect of scientific investigation (Elliott & Busse, 1993). 
Treatment plans implemented with poor integrity make it difficult to draw accurate inferences about the 
relationship between an intervention and behavior change (Gutkin, 1993). In many consultation cases, 
absent or weak treatment effects might be the result of an intervention’s poor integrity, despite its 
demonstrated empirical support (Cautilli, Tillman, Axelrod, Dziewolska, Hineline, 2006; Cautilli, Riley- 
Tillman, Axelrod, & Hineline, 2005). According Gresham (1989), “Many failures in consultation and 
interventions probably can be attributed to the fact that intervention plans are not implemented as 
intended” (p. 137). Thus, it is essential that treatment integrity information be collected when 
implementing consultation-derived interventions in order to distinguish between ineffective treatments 
and potentially effective interventions implemented with poor integrity (Gresham, 1989; Gutkin, 1993). 


Treatment Integrity in Research and Practice 

Although the importance of treatment validity has been acknowledged in the literature, this 
construct has largely been ignored in research and practice (Erchul & Shulte, 1996; Witt, Noell, LaFleur, 
& Mortenson, 1996; Lane, Bocian, MacMillan, & Gresham, 2004). Relatively few intervention studies 
have monitored or systematically assessed treatment integrity (Gresham, 1989; Gresham, 2004). For 
example, only 20% of behavioral intervention studies published in the Journal of Applied Behavior- 
Analysis from 1968 to 1980 presented data concerning treatment integrity (Peterson, Homer, & 
Wonderlich, 1982). Likewise, 14.4% of school-based and 16% of child-based behavioral intervention 
studies published in the Journal of Applied Behavior Analysis between the years 1 980 to 1 990 provided 
treatment integrity information (Gresham, Gansle, & Noell, 1993; Gresham, Gansle, Noell, Cohen, & 
Rosenblum, 1993). A survey of intervention articles published in three major learning disabilities journals 
from 1995 to 1999 found that only 18.5% of the studies measured and reported data on treatment integrity 
(Gresham, MacMillan, Beebe-Frankenberger, & Bocian, 2000). Similarly, approximately 25% of research 
articles published in the Journal of Applied Behavior Analysis from 1 996 to 2000 reported treatment 
integrity measures (Progar, Perrin, DiNovi, & Bruce, 2001). Most recently, a review of intervention 
studies conducted with children with autism published in recognized behavioral journals (e.g., Journal of 
Applied Behavior Analysis) between the years 1993-2004 found that just 18% actually assessed and 
reported treatment integrity data (Wheeler, Baggett, Fox, & Blevins, 2006). 


Based on the existing literature, it seems clear that attention to treatment integrity is the exception 
rather than the rule (Gresham et al., 1993a). This apparent double standard in consultation research and 
practice is troubling. For example, practitioners are expected to give careful attention to the design and 
implementation of interventions, selection of outcome measures, and the accuracy with which the 
outcome data are collected (Lane et al., 2004). Yet, the same consideration is not given to the 
implementation of school-based interventions (Gresham et al., 2000). Consultants frequently assume that 
a consultee's good intentions and verbal agreement will result in fidelity to their intervention plans. 
Research also suggests that in a majority of cases, school psychologists either fail to monitor plans or do 
not include an evaluation component in their consultation practice. When evaluations are conducted, 
informal teacher verbal reports tend to be the most common form of outcome measure (Bramlett et al., 
2002). 

Purpose 

While it seems reasonable to expect that behavioral consultation practice should pay more, not 
less attention to the assessment of treatment integrity, the model addresses this construct in no more than 
a cursory manner (Erchul and Shulte, 1996; Gutkin, 1993; Witt, Gresham, & Noell, 1996). Although brief 
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contacts are made with the consultee during treatment implementation, the traditional behavioral 
consultation process does not provide a structured method of monitoring treatment integrity (Kratochwill 
& Bergan, 1990). The primary purpose of this article is to propose an expansion of the traditional 
behavioral consultation model that provides school-based consultants with a method of assessing and 
monitoring fidelity to intervention plans. 

Assessing Treatment Integrity 

There are several methods school psychologists can use to document the extent to which 
behavioral interventions agreed upon during consultation are being implemented as intended by the 
change agent (e.g., teacher or parent). Although systematic observation is the most direct means of 
assessing treatment integrity, this procedure is vulnerable to reactivity effects and tends to be a labor 
intensive activity which is not always feasible given the time constraints and logistical problems 
encountered in real world school settings (Wilczynski, Mandal, & Fuslier, 2000). There are, however, less 
direct methods that can be utilized to assess and monitor the fidelity of treatment plans implemented in 
consultation practice. They include (a) consultee self-reporting, (b) permanent products, (c) behavioral 
interviews and (d) feedback from consultants (Lane et al., 2004). 

Self-Reports 

Self-reports of treatment integrity involve having consultees report the extent to which they have 
implemented each component of the intervention. Table 1 provides an example of how treatment integrity 
can be assessed using a self-report method for a cognitive behavior modification intervention. As shown, 
the teacher reported that five out of six self-management intervention components were implemented on 
Monday, Tuesday, and Friday, resulting in a daily integrity rating of 83%, (b) three out of six 
components were applied on Wednesday and Thursday, indicating 50% integrity each day, and (c) 
components one and two were implemented with 100% fidelity over the course of the week. In contrast, 
components five and six were implemented with only 20% accuracy. Although consultee self-reports can 
provide school psychologists with important treatment integrity data, not all interventions are well-suited 
to this assessment method. Moreover, this technique may overestimate the level of treatment fidelity and 
produce inaccurate reports (Gresham et al., 2000). 

Table 1. Example of a Treatment Integrity Checklist 


Treatment Integrity Checklist 
Cognitive Behavior Modification Plan 


Student: Teacher: Grade Level: 

Directions'. Please complete this form each day. Record a “Y” if the component was 
implemented; if the component was not implemented, record an “N”. 

Intervention components Day 

M T W T F 
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Reviewed behavior goal(s) with student. 

Y 

Y 

Y 

Y 

Y 

Cued student to self-monitor and record response. 

Y 

Y 

Y 

Y 

Y 

Compared ratings with student. 

Y 

Y 

N 

Y 

Y 

Provided verbal praise for accurate ratings. 

Y 

Y 

Y 

N 

Y 

Gave reward when behavioral goal was met. 

Y 

N 

N 

N 

Y 

Sent behavior recording form to parent. 

N 

Y 

N 

N 

N 


Note. Adapted from “Monitoring Treatment Integrity: An Alternative to the ‘Consult and 
and Hope’ Strategy in School-Based Behavioral Consultation,” by Lee A. Wilkinson, 2006, 
School Psychology International, 27, p. 431. Copyright 2006 by Sage Publications. Adapted with 
permission. 

Permanent Products 

Permanent product assessment refers to a product that is generated as the result of an intervention 
and evaluated to determine the extent to which a corresponding component was implemented as intended 
(Lane et al., 2004). To illustrate, a permanent product could be used to monitor the treatment integrity of 
the intervention used in the previous self-report example (Table 1). In this instance, treatment integrity 
would be assessed by examining the student’s self-monitoring recording form and/or home-school note 
each day to determine fidelity to this intervention component. Another example might involve an 
intervention to increase academic productivity. Classroom worksheets or assignments would be collected 
and evaluated by the consultant to determine accuracy and the level of task completion. Permanent 
products have the advantage of being less time consuming, more efficient, less reactive and potentially 
more accurate than other assessment methods (Gresham et al., 2000; Lane et al., 2004). 


Inten’iew and Performance Feedback 

Research indicates that a combination of treatment monitoring and performance feedback can 
increase the level of treatment implementation and enhance student behavioral outcomes (Jones, 
Wickstrom, & Friman, 1997; Mautone, Luiselli, & Handler, 2006; Mortenson & Witt, 1998; Noell et al., 
2005; Noell, Duhon, Gatti, & Connell, 2002; Noell, Witt, LaFleur, Mortenson, Ranier, & LeVelle, 2000; 
Noell, Witt, Gilbertson, Ranier, & Freeland, 1997; Witt, et al., 1997). Expanding the traditional 
behavioral consultation model to include a structured treatment monitoring stage, rather than only brief 
informal contacts, should be an effective means by which to provide valuable performance feedback and 
supportive assistance to consultees. (Codding, Feinberg, Dunn, & Pace, 2005; Knoff, Hines, & Kromery, 
1995; Wilkinson, 2005). 

Table 2 depicts the stages and objectives of the expanded behavioral consultation model. The 
primary objective of the treatment monitoring interview (TMI) is to increase the strength of the treatment 
plan by providing direct support and performance feedback (Butler, Weaver, Doggett, & Watson, 2002). 

A semi-structured interview (see Appendix) is conducted with the consultee(s) to (a) review the student’s 
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behavioral progress, (b) identify barriers or obstacles to plan implementation, and (c) determine the need 
for plan modification. A performance feedback session is then conducted to (a) analyze treatment 
integrity data (permanent products and self-report checklists), (b) provide positive praise for completed 
components and corrective feedback for steps omitted or incorrectly implemented, (c) address consultee 
questions and concerns, (d) determine the need for further training or support, and (e) reaffirm the 
consultee’s commitment to implementing the plan. 

Table 2 . The Expanded Behavioral Consultation Model 


Problem Identification Interview (PII) 

1 . Define the target problem(s) in observable terms. 

2. Identify important environmental conditions that influence behavior. 

3. Agree on a goal for behavioral change. 

4. Establish a procedure for data-collection. 

Problem Analysis Interview (PAI) 

1 . Analyse baseline data. 

2. Identify antecedent, consequent, and sequential conditions. 

3. Design a treatment plan strategy. 

Treatment (Plan) Implementation 

1 . Implement treatment plan. 

2. Continue data-collection procedure. 

Treatment Monitoring Interview (TMI) 

1 . Increase fidelity to the intervention plan. 

2. Review pennanent products and self-reports. 

3. Provide data-based feedback and support. 

4. Determine need for additional training. 

Treatment Evaluation Interview (TEI) 

1 . Evaluate treatment plan effectiveness. 

2. Assess social validity and consultant effectiveness. 

3. Discuss continuation, modification, or termination of the plan. 


Note. Adapted from ‘Using behavioral consultation to reduce challenging behavior in the 
classroom,’ by Lee A. Wilkinson, 2003, Preventing School Failure, 47, p. 101. Copyright 1993 
by Heldref Publications. Adapted with permission. 
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Treatment Monitoring Interview (TMI) 

Child’s Name: Date: 

Teacher’s Name: Age: 

School: Grade: 

The consultant should question and/or comment on all of the following: 

OPENING SALUTATION 

GENERAL STATEMENT/QUESTION REGARDING PLAN IMPLEMENTATION 

a. How successful were you in implementing the intervention as discussed during our last 
meeting? Please describe. 

Record responses: 


b. Is the plan producing the desired change in target behavior? What changes have you 
observed in the student's academic/behavioral performance (positive or negative)? 

Record Responses: 


OBSTACLES TO PLAN IMPLEMENTATION 

a. Have you encountered any barriers to implementing the plan? If so, please specify. 
What can we do to overcome them? 
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Record Responses: 


b. Does the plan need to be altered or changed in some way in order to effective? 
If yes, how? Will revising the plan help you to implement it more consistently? 

Record Responses: 


DATA BASED (PERFORMANCE) FEEDBACK 

a. Let’s look at the data you’ve collected so far so we can evaluate the student’s progress 
(Summarize/graph student performance data and evaluate pennanent products). 

Record Responses: 


b. Let’s review your treatment plan (self-report) ratings and discuss any of the steps 
you had difficulty implementing (provide praise for completed components). If you 
are having problems, do you need more training or assistance in order to implement 
the plan? If so, please describe. 

Record Responses: 


NEXT APPOINTMENT 

When can we meet again to discuss the data and determine whether the plan has been effective? 
Date: 
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Day: 

Time: 


Implications for Consultation Practice 

Research challenges the assumption that treatment integrity is being assessed and monitored in 
behavioral consultation and intervention practice (Jones et al., 1997). Knowing that an intervention is 
effective and understanding how to use it does not guarantee its accurate implementation and use 
(Wickstrom et al., 1998). Likewise, including elements of rational persuasion (e.g., discussing the 
importance of the intervention and addressing possible objections) in brief meetings with consultees are 
inadequate to support consistent treatment implementation (Truscott et al., 2003). Expanding the 
behavioral consultation model to include a treatment monitoring interview (TMI) offers consultants a 
practical and effective method of monitoring treatment integrity. Practitioners and researchers will find 
the interview procedures and objectives presented in this article helpful in coordinating the intervention 
process and providing efficient follow-up on treatment plan implementation. Treatments implemented 
with poor integrity can prompt consultants to make adjustments to treatment plans and expend greater 
time and effort on the plan implementation stage of consultation (Gresham et al., 1993). Information 
obtained during the TMI can also be used to determine whether additional procedures or support are 
needed for training consultees to implement the intervention properly and to assist in generalization of 
consultation-based skills (Tillman, 2000). 


Limitations 

While including a formal TMI in the behavioral consultation model holds promise for enhancing 
treatment integrity, there are limitations inherent in this approach. For example, the consultation process 
conceptualized here is likely to be a labor-intensive activity. The four consultation interviews require a 
substantial investment of time from both the consultant and consultee(s). This high level of active 
participation and commitment required by the expanded BC model might be unrealistic for many 
educators given their classroom schedules and limited planning time (Wilczynski et al., 2000). Likewise, 
more frequent feedback than that provided by the TMI may be required to sustain treatment 
implementation, further increasing logistic demands (Jones et al., 1997; Noell et al., 2005). From a cost- 
benefit perspective, procedural aspects of BC may need adjustment to accommodate these empirical 
barriers. For example, it might be necessary to streamline the process and reduce duplication to improve 
efficiency. Another important limitation involves the use of consultee self-reports to monitor treatment 
plan implementation. This method can be influenced by social desirability and may inflate estimated 
levels of integrity when compared to direct observation and permanent product data (Gresham et al., 
2000; Lane et al., 2004). 


Research Directions 

Empirically validated procedures for assessing and improving treatment implementation are not 
well developed and remain a critical research need. Future research should investigate adaptation of the 
TMI to consultation practice to determine its effectiveness in facilitating treatment integrity. Additionally, 
research should examine the agreement between direct observation and self-reports/permanent products to 
evaluate the robustness of these indirect assessment methods. It is also unclear what levels of treatment 
integrity are necessary to ensure positive outcomes. Treatment integrity and student performance might be 
measured at varying points during treatment implementation to determine the extent to which continued 
direct support and performance feedback is necessary to maintain an adequate level of treatment integrity 
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(Progar et. al., 2001). Lastly, research should include the experimental manipulation of both treatment 
acceptability and integrity as dependent variables to more fully understand their roles in behavior change 
(Noell et al., 2002). 

Conclusion 

The fidelity with which consultative-derived intervention plans are implemented with students is 
an essential component of consultation practice (Gresham et al., 2000). As problem-solving consultation 
is more widely adopted by the school psychology community, practitioners must recognize the 
importance of treatment integrity as a defining feature of successful intervention (Erchul & Schulte, 
1996). Expanding the behavioral consultation model to include the treatment monitoring interview (TMI) 
described in this article represents an initial step in developing the structure and technology necessary to 
address one of the most critical aspects of school-based consultation practice: ensuring the accurate 
implementation of our behavioral interventions. 
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